Code Coverage and Quality
Part 2, Chapter 10
In this chapter, we'll add tools for checking code coverage and quality to the project.
Code Coverage
Code coverage is the measure of how much code is executed during testing. By adding code coverage to your test suite, you can find areas of your code not covered by tests.
Coverage.py is a popular tool for measuring code coverage in Python-based applications. Now, since we're using pytest, we'll integrate Coverage.py with pytest using pytest-cov.
Add pytest-cov to the requirements.txt file:
pytest-cov==3.0.0
Next, we can configure the coverage reports in a .coveragerc file. Add this file to the "project" directory, and then add the following config to exclude the tests from the coverage results and enable branch coverage measurement:
[run]
omit = tests/*
branch = True
Update the containers:
$ docker-compose up -d --build
Run the tests with coverage:
$ docker-compose exec web python -m pytest --cov="."
You should see something similar to:
=============================== test session starts ===============================
platform linux -- Python 3.10.1, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /usr/src/app
plugins: anyio-3.4.0, cov-3.0.0
collected 6 items
tests/test_ping.py . [ 16%]
tests/test_summaries.py ..... [100%]
---------- coverage: platform linux, python 3.10.1-final-0 -----------
Name Stmts Miss Branch BrPart Cover
----------------------------------------------------------
app/__init__.py 0 0 0 0 100%
app/api/__init__.py 0 0 0 0 100%
app/api/crud.py 15 0 2 0 100%
app/api/ping.py 6 0 0 0 100%
app/api/summaries.py 20 0 2 0 100%
app/config.py 13 2 2 0 87%
app/db.py 17 7 2 1 58%
app/main.py 18 3 0 0 83%
app/models/__init__.py 0 0 0 0 100%
app/models/pydantic.py 5 0 4 0 100%
app/models/tortoise.py 9 1 2 0 91%
----------------------------------------------------------
TOTAL 103 13 14 1 88%
================================ 6 passed in 0.56s ================================
Want to view an HTML version?
$ docker-compose exec web python -m pytest --cov="." --cov-report html
The HTML version can be viewed within the newly created "htmlcov" directory. With it, you can quickly see which parts of the code are, and are not, covered by a test.
$ open project/htmlcov/index.html
Add this directory along with the .coverage file to the .gitignore and .dockerignore files.
Just keep in mind that while code coverage is a good metric to look at, it does not measure the overall effectiveness of the test suite. In other words, having 100% coverage means that every line of code is being tested; it does not mean that the tests handle every scenario.
Just because you have 100% test coverage doesn’t mean you're testing the right things.
Code Quality
Linting is the process of checking your code for stylistic or programming errors. Although there are a number of commonly used linters for Python, we'll use Flake8 since it combines two other popular linters -- pep8 and pyflakes.
Add Flake8 to the requirements.txt file:
flake8==4.0.1
To configure Flake8, add a setup.cfg file to the "project" directory:
[flake8]
max-line-length = 119
This sets the maximum allowed line length to 119.
Update the containers:
$ docker-compose up -d --build
Run Flake8:
$ docker-compose exec web flake8 .
Were any errors found?
Correct any issues before moving on.
Next, let's add Black, which is used for formatting your code so that "code looks the same regardless of the project you're reading". This helps to speed up code reviews. "Formatting becomes transparent after a while and you can focus on the content instead."
Add the dependency to the requirements file:
black==21.12b0
Update the containers, and then run Black:
$ docker-compose up -d --build
$ docker-compose exec web black . --check
Since we used the check option, you should see the status:
would reformat app/api/ping.py
would reformat app/main.py
would reformat app/api/summaries.py
would reformat tests/test_summaries.py
Oh no! 💥 💔 💥
4 files would be reformatted, 11 files would be left unchanged.
Try running it with the diff option as well before applying the changes:
$ docker-compose exec web black . --diff
$ docker-compose exec web black .
Finally, let's add isort to the project as well to quickly sort all our imports alphabetically and automatically separate them into sections.
Again, add the dependency:
isort==5.10.1
Run it with the check-only and diff options:
$ docker-compose up -d --build
$ docker-compose exec web isort . --check-only
$ docker-compose exec web isort . --diff
Then, apply the changes:
$ docker-compose exec web isort .
Verify one last time that Flake8, Black, and isort all pass:
$ docker-compose exec web flake8 .
$ docker-compose exec web black . --check
$ docker-compose exec web isort . --check-only
Commit your code.
✓ Mark as Completed